Surface Majorana flat bands in j = 3/2 superconductors with singlet-quintet mixing
Yu Jiabin, Liu Chao-Xing
Department of Physics, the Pennsylvania State University, University Park, PA 16802, USA

 

† Corresponding author. E-mail: cxl56@psu.edu

Abstract

Recent experiments [Science Advances 4 eaao4513 (2018)] have revealed the evidence of nodal-line superconductivity in half-Heusler superconductors, e.g., YPtBi. Theories have suggested the topological nature of such nodal-line supercon-ductivity and proposed the existence of surface Majorana flat bands on the (111) surface of half-Heusler superconductors. Due to the divergent density of states of the surface Majorana flat bands, the surface order parameter and the surface impurity play essential roles in determining the surface properties. We study the effect of the surface order parameter and the surface impurity on the surface Majorana flat bands of half-Heusler superconductors based on the Luttinger model. To be specific, we consider the topological nodal-line superconducting phase induced by the singlet-quintet pairing mixing, classify all the possible translationally invariant order parameters for the surface states according to irreducible representations of C3v point group, and demonstrate that any energetically favorable order parameter needs to break the time-reversal symmetry. We further discuss the energy splitting in the energy spectrum of surface Majorana flat bands induced by different order parameters and non-magnetic or magnetic impurities. We propose that the splitting in the energy spectrum can serve as the fingerprint of the pairing symmetry and mean-field order parameters. Our theoretical prediction can be examined in the future scanning tunneling microscopy experiments.

1. Introduction

Recent years have witnessed increasing research interests in half-Heusler compounds (RPdBi or RPtBi with R a rare-earth element)[2] due to their non-trivial band topology,[316] magnetism,[1725] and unconventional superconductivity.[1,17,2022,2633] Half-Heusler superconductors (SCs) are of particular interest because of the low carrier density (1018–1019 cm−3), the power-law temperature dependence of London penetration depth, and the large upper critical field. Furthermore, it was theoretically proposed that electrons near Fermi level in half-Heusler SCs possess total angular momentum j = 3/2 as a result of the addition of the 1/2 spin and the angular momentum of p atomic orbitals (l = 1).[1,34] Therefore, half-Heusler SCs provide a great platform to study the superconductivity of j = 3/2 fermions. Such j = 3/2 fermions were also studied in anti-perovskite materials[35] and the cold atom system.[36,37] Due to the j = 3/2 nature, the spin of Cooper pairs can take four values: S = 0 (singlet), 1 (triplet), 2 (quintet), and 3 (septet), among which quintet and septet Cooper pairs cannot appear for spin-1/2 electrons.

In order to understand the unconventional superconductivity, various pairing states were proposed, including mixed singlet-septet pairing,[1,34,38,39] mixed singlet-quintet pairing,[4042] s-wave quintet pairing,[34,39,43,44] d-wave quintet pairing,[44,45] odd-parity (triplet and septet) parings,[4548] et al.[46,49] In particular, references [1,34,3842] proposed that the power-law temperature dependence of London penetration depth can be explained by topological nodal-line superconductivity (TNLS) generated by the pairing mixing between different spin channels. In particular, it has been shown that two types of pairing mixing states, the singlet-quintet mixing and singlet-septet mixing, can both give rise to nodal lines in certain parameter regimes.

In this work, we focus on the singlet-quintet mixing, which was proposed in Ref. [40]. As a consequence of TNLS, the Majorana flat bands (MFBs) are expected to exist on the surface perpendicular to certain directions. Such surface MFBs (SMFBs) are expected to show divergent quasi-particle density of states (DOS) at the Fermi energy and thus can be directly probed through experimental techniques, such as scanning tunneling microscopy (STM).[50] Due to the divergent DOS, certain types of interaction[5154] and surface impurities[5557] are expected to have a strong influence on SMFBs. This motivates us to study the effect of the interaction-induced surface order parameter and the surface impurity on the SMFBs of the superconducting Luttinger model with the singlet-quintet mixing. Specifically, we classify all the mean-field translationally invariant order parameters of the SMFBs according to the irreducible representations (IRs) of C3v group, identify their possible physical origins, and show their energy spectrum by calculating the corresponding DOS. We find that the order parameter needs to break the time-reversal (TR) symmetry in order to either gap out the SMFBs or convert the SMFBs to nodal-lines or nodal points. We also study the quasi-particle local DOS (LDOS) of SMFBs with a surface charge impurity or a surface magnetic impurity (whose magnetic moment is perpendicular to the surface), and show that the peak splitting induced by different types of impurities can help to distinguish the pairing symmetries and surface order parameters.

The rest of the paper is organized as follows. In Sections 2 and 3, we briefly review the superconducting Luttinger model with singlet-quintet mixing and illustrate the symmetry properties of SMFBs. In Section 4, we classify all the mean-field translationally invariant order parameters according to the IRs of C3v and identify their physical origin. We also calculate the energy spectrum and DOS of SMFBs with different order parameters. In Section 5, the impurity effect on the LDOS of MFBs with/without the surface order parameter is discussed. Finally, our work is concluded in Section 6.

2. Model Hamiltonian

The model that we used to generate MFBs in this work is the same as that studied in Ref. [40], which describes the superconductivity in the Luttinger model with mixed s-wave singlet and isotropic d-wave quintet channels. The Bogoliubov–de-Gennes (BdG) Hamiltonian in the continuous limit reads

where is the Nambu spinor and are creation operators of j = 3/2 fermionic excitations. The term

consists of the normal part h(k) that is the Luttinger model[4,40,58,59]

and the paring part Δ(k) that contains s-wave singlet and isotropic d-wave quintet channels

where μ is the chemical potential, c1, c2 indicate the strength of the centrosymmetric spin–orbital coupling (SOC) which is the coupling between the orbital and the 3/2-“spin”, d-wave cubic harmonics gk,i and five Γ matrices are shown in Appendix A, Δ0,1 are order parameters of the singlet and quintet channels, respectively, a is the lattice constant of the material, and γ = −Γ1Γ3 is the TR matrix. The coexistence of the two order parameters is allowed by their same symmetry properties.[40,6066]

Before demonstrating the SMFB generated by Eq. (1), we first discuss the symmetry properties of Hamiltonian H. As discussed in Ref. [40], H has TR symmetry, and its point group is O(3) or Oh for c1 = c2 or c1c2, respectively. Due to the coexistence of TR and inversion symmetries, the Luttinger model h(k) has two doubly degenerate bands ξ±(k) = k2/(2m±) − μ, where are effective masses of two bands, and . In addition, the particle–hole (PH) symmetry can be defined as and for the BdG Hamiltonian, where C = τx with τx the Pauli matrix for the PH index. Combining the PH and TR symmetries, we have the chiral symmetry −χhBdG(k)χ = hBdG(k), where χ = i𝒯C* and 𝒯 = diag(γ, γ*) is the TR matrix on the Nambu bases. The representations of other symmetry operators are shown in Appendix B.

3. Surface Majorana flat bands

In this work, we choose μ < 0, m < 0, c1c2 > 0, and focus on the case where c1c2, m± < 0, and SMFBs exist on the (111) surface.[40] To solve for SMFBs, we consider a semi-infinite configuration (x < 0) of Eq. (1) along the (111) direction with an open boundary condition at the x = 0 surface, where x labels the position along (111). In this case, the point group is reduced from Oh to C3v, which is generated by three-fold rotation along the (111) direction and the mirror operation perpendicular to the direction. Although the translational invariance along (111) is broken, the momentum k that lies inside the (111) plane is still a good quantum number, and we define k‖,1 and k‖,2 along the and directions, respectively.

Following Ref. [40], we find that SMFBs can exist in certain regions of the surface Brillouin zone, denoted as A in Fig. 1, and originate from the non-trivial one-dimensional AIII bulk topological invariant (Nw = ±2). At each kA, the semi-infinite model has two orthonormal solutions of zero energy that are localized near the x = 0 surface and have the same chrial eigenvalues, coinciding with the bulk topological invariant Nw = ±2.

Fig. 1. Distribution of SMFBs for |2m|c1 = 0.8, , and , where , and . The surface zero modes in red (orange) regions have 1 (−1) chiral eigenvalue, and ’s are labeled according to the convention. The dashed lines are given by k‖,1 = 0 and k‖,2 = ±k‖,1/2, where the surface zero modes cannot exist.

We label the creation operators for the two zero-energy solutions at kA as with i = 1,2, and they satisfy the anti-commutation relation

The subscript i = 1,2 of can be regarded as the pseudospin index, since can furnish the same representation of TR, , and operators as a two-dimensional j = 1/2 fermion by choosing the convention

where 𝒯b = iσ2, , and σ1,2,3 are the Pauli matrices for the pseudospin of SMFBs. Since the chiral matrix χ commutes with any operation in C3v and anti-commutes with TR operation, has the same the chiral eigenvalue as , but opposite to , where RC3v. As a result, the surface zero-energy modes cannot exist on three lines parametrized by k‖,1 = 0 and k‖,2 = ±k‖,1/2, dividing the region A into six patches as shown in Fig. 1. Since the chiral eigenvalues of the zero-energy modes in one patch are the same, we can label each patch as with lχ = ± for the chiral eigenvalues ±1 and lc = 1,2,3 marking three patches related by rotation. Furthermore, we choose to be symmetric under k‖,2 → −k‖,2, i.e., the mirror operation perpendicular to . Due to the PH symmetry, the surface zero modes at ±k are related by

where is the chiral eigenvalue of the zero modes at k, i.e., for kA± with . TR and C3v symmetries imply and with RC3v (see Appendix C for details).

4. Mean-field order parameters of surface Majorana flat bands

Due to the divergent DOS, the interaction may result in the nonvanishing order parameters at the surface and give rise to a gap of SMFBs. In this section, we study the possible mean-field order parameters on the (111) surface that preserve the in-plane translation symmetry. We find that the order parameters must break the TR symmetry in order to gap out the SMFB; all the TR-breaking surface order parameters are classified based on the IRs of C3v and their physical origins are identified. Then, to the leading order approximation where the surface order parameters are independent of k in each of the surface mode regions, we find that the SMFBs can be generally gapped out by these order parameters, and the gapless modes are only possible for certain IRs with certain finely tuned values of parameters. We further study the LDOS structure of SMFBs in the presence of various order parameters and find that the splitting patterns of the LDOS peak can be used to distinguish different order parameters as summarized in Figs. 2 and 4.

Fig. 2. The LDOS on the (111) surface as a function of the energy (E/|μ|) (a) without any order parameters, (b) with the A1 order parameter, (c) with the A2 order parameter, and (d) with the E order parameter. Due to the PH symmetry, only non-negative-energy half of the LDOS is physical. The broadening of each peak is plotted via Gaussian distribution with standard deviation being 10−3. The parameters choices for each order if exist are m1/|μ| = 0.05 and m2/|μ| = 0.1 for the A1 order parameter (16), m3/|μ| = 0.05 and m4/|μ| = −0.1 for the A2 order parameter (17), and 𝒎5/|μ| = (0.01,0.02), 𝒎6/|μ| = (0.03,0.04), 𝒎7/|μ| = (0.05,0.06), and 𝒎8/|μ| = (0.07,0.08) for the E order parameter (18). Here we do not show the numbers on the vertical axis[68] since only the position of LDOS peak can be probed in the STM experiments.
Fig. 3. The LDOS as a function of the energy (E|μ|) with surface impurities: (a)–(d) a surface charge impurity and (e)–(h) a surface magnetic impurity. The four columns from left to right correspond to no order parameters, A1 order parameter, A2 order parameter, and E order parameter, respectively. The broadening of each peak and the parameters choices for the orders if exist are the same as those in Fig. 2. The potential form of the charge or magnetic impurity is shown in Appendix F. The numbers on the vertical axes are again omitted.
Fig. 4. This graph shows how the number of LDOS peaks shown in Figs. 2 and 3 is determined by the symmetry. The solid black lines indicate the LDOS peaks. A1, A2, and E stand for the surface order parameters, and Vc and Vm denote the charge and magnetic impurities, respectively. “Deg” indicates the symmetry protected degeneracy of each LDOS peak, except the case marked by (*) where only half of the eight peaks have the double degeneracy. If Deg > 1, the line below shows the crucial symmetries that account for the degeneracy. Here Π means odd mirror parity, T means the translational invariance, and the origin for the rotation C3 or mirror Π is located at the impurity center. The red lines crossing the symmetry operations indicate the breaking of the corresponding symmetries.
4.1. Symmetry classification and physical origin

The general form of translationally invariant fermionbilinear terms for SMFBs can be constructed as

where m(k) is a 2 × 2 Hermitian matrix. The PH symmetry makes m(k) satisfy m(k) = −σ2 mT(−k2 up to a shift of the ground state energy based on Eq. (7), while the TR symmetry requires according to Eq. (6). As a result, the combination of PH and TR symmetries, which is equivalent to the chiral symmetry, leads to m(k) = 0, indicating that the existence of a non-vanishing fermion bilinear term m(k) for the SMFBs requires the breaking of the TR symmetry, i.e.,

As the C3v point group symmetry can also be spontaneously broken by these fermion-bilinear terms, we can further classify these TR-breaking order parameters according to the IR of C3v, of which the character table (Table A1) is shown in Appendix A. Since C3v has three IRs A1, A2, and E, equation (8) can be expressed as the linear combination of the three corresponding parts

Table 1.

The irreducible representations of C3v generated by , σl, ρl, or λl with their parities under TR, PH, and chiral operation. The transformation of is defined as , the transformation of σl is , the transformation of ρl is , and the transformation of λl is , where R = −1, −1, 1,C3, Π; Rb = iσ2K, σ2K, 1, C3,b, Πb; Rχ = 𝒯χK, CχK, χχ,C3,χ, Πχ; and Rc = 𝒯cK, CcK, χc,C3,c, Πc for TR, PH, χ, C3, and Π, respectively, and K is the complex conjugate operation. The parity α = ± is defined as XαX under the operation of TR, PH, or χ and thus being TR, PH, and χ symmetric correspond to α = +,−,−, respectively. lχ = ±, lc = 1,2,3, and is equal to 1 if and 0 otherwise. and .

.

Here the A1 term preserves C3v symmetry, and the A2 term preserves symmetry but has odd mirror parity. The E term has the expression mE(k) = a1mE,1(k) + a2mE,2(k) with (mE,1(k),mE,2(k)) a two-component vector that can furnish a E IR; it breaks the entire C3v symmetry except for some special values of (a1, a2), e.g., one of the three mirrors is preserved but is broken for , or .

Next we illustrate the physical origin of each term in Eq. (10) by considering the following on-site mean-field Hamiltonian that is independent of k:

where and . Equation (9) can be obtained by projecting the above Hamiltonian onto the surface, and such projection does not change the symmetry properties. Since m(k) must be TR odd in order to be non-vanishing, it requires and to be TR odd. Then, the TR-breaking and can be classified into different IRs of C3v:

where and can only give rise to mβ (k) in Eq. (10) with β = A1,A2,E (see Appendix D for details). Concretely, we have

where ni’s are listed in Table A2 of Appendix A, and ζj(x)’s are real. Physically, n0γ corresponds to the singlet pairing, n1γ, 𝒏6γ, and 𝒏7γ generate quintet pairings, and n4, n8,1, and n8,2 give FM in (111), , and directions, respectively. Since n2,n3,n5,𝒏9, and 𝒏10 can be represented by the linear combinations of with the septet spin tensor S3m (m = −3,−2,…,3), we dub these terms the spin-septet order parameters. As a summary, can be generated by the singlet pairing, the quintet pairing, and the spin-septet order parameter; can be generated by (111)-directional ferromagnetism (FM) and the spin-septet order parameter; mE(k) can be generated by the quintet pairing, the FM perpendicular to the (111) direction, and the spin-septet order parameter.

4.2. Surface local density of states

In the following, we focus on the order parameters that are independent of k in every one of six surface mode regions ’s. In this case, equation (10) can be expanded as

where is real, if and 0 otherwise, and σl labels the Pauli matrix for pseudospin. Then, for any symmetry transformation of m(k), we can convert the transformation of pseudospin index and k dependence of m(k) to the transformation of σl and , respectively. Based on the symmetry transformation, we can classify and σl according to the IRs of C3v and the parities under TR, PH, and χ, as shown in the top and second top parts of Table 1, respectively. The symmetry classification of TR-odd terms in m(k) can be obtained by the tensor product of σl and , as shown in Table A3 of Appendix A with various terms labeled by Ni’s. As a result, we have the following general expressions of the order parameters in different IRs of C3v:

Here all mj’s are real.

With Eqs. (16)–(18), we next discuss the energy spectrum and LDOS of SMFBs after including these order parameters. Due to the PH symmetry, only half of the energy spectrum (non-negative energy part) gives the quasi-particle LDOS of SMFBs. However, it is more convenient to study the full spectrum, since the LDOS, which is probed by the tunneling conductance of STM, must symmetrically distribute with respect to the zero energy in experiments.[67] Since the order parameters in each patch are k-independent, we choose the mode at the geometric center of each patch as the representative mode. In the following, we only consider the representative modes and use the term “degeneracy” to refer to the extra degeneracy determined by the symmetry, excluding the large degeneracy given by the flatness of the dispersion in each patch. For convenience, we define the creation operator to label the representative mode in the patch with the pseudo-spin index i. Since only the uniform order parameters are considered, lχ and lc are good quantum numbers, while different pseudo-spin components (the σl part) are typically coupled by the order parameter m(k). Thus, we introduce the band index s = ± and label the eigen-mode as with

the eigen-equation.

Without any order parameters, all these 12 modes, including 6 patches and 2 pseudospin components, are degenerate and thus the SMFBs have a zero-bias peak for LDOS, as shown in Fig. 2(a). For the A1 order , the eigenenergies are given by , and once |m1| ≠ |m2|, all the zero energy peaks will be split for SMFBs. As a result, the LDOS of the A1 order parameter typically has 4 peaks shown in Fig. 2(b). This peak structure of LDOS can be understood from symmetry consideration. Due to the breaking of TR symmetry, as well as the chiral symmetry, we only need to consider the point group symmetry C3v. As mentioned before, any operation in C3v does not change the lχ index, and since the A1 order parameter is C3v invariant, the band index s cannot be changed either. The C3 rotation only transforms the lc = 1,2,3 index counter-clockwise, resulting in the three-fold degeneracy among the eigen-modes with the same s and lχ. On the other hand, Π interchanges lc = 1,2 and makes sure that has the same energy as , meaning that Π does not give extra constraints compared with C3. Thus, there are 12/3 = 4 peaks in the LDOS of the A1 order parameter with each peak of 3-fold degeneracy. For the A2 order parameter , the eigen-energies are given by , leading to 2 peaks in the LDOS (Fig. 2(c)), resulted from the six-fold degeneracy of each eigen-energy due to the symmetry. Among the six-fold degeneracy, three-fold degeneracy is due to the translational invariance and C3 symmetry as the A1 order parameter, meaning that , and have the same energy. The remaining double degeneracy originates from the combination of the odd mirror parity of the A2 order parameter and the PH symmetry, i.e., . This combined symmetry does not change the band index s, but transforms lχ as + ↔ − and lc as 1 2. As a result, with fixed s and lc also have the same energy, giving the extra double degeneracy. For the E order parameter mE (k), the eigen-energies are , where m¯1=m5,1232m5,2m¯2=m5,12+32m5,2,m¯3=m5,1m¯1=[ (32m6,1+m6,22) 2+(m7,1+m8,12+32m8,2)2+(m7,232m8,1+m8,22)2 ]1/2,m¯2=[ (32m6,1+m6,22) 2+(m7,1+m8,1232m8,2)2+(m7,2+32m8,1+m8,22)2 ]1/2,m¯3=[m6,22+(m7,1+m8,1)2+(m7,2m8,2)2]1/2.

Therefore, all the modes are typically split for the E order and the corresponding LDOS generally has 12 peaks shown in Fig. 2(d).

We would like to mention that if including the momentum dependence of the surface order parameter in each surface-mode region, it can broaden the LDOS peaks in Fig. 2. In addition, the momentum dependence may also lead to the existence of arcs of surface zero modes in certain small parameter regions as discussed in Appendix E.

5. Impurity effect

In this section, we will study the effect of surface non-magnetic and magnetic impurities. The effect of non-magnetic impurity on SMFBs in the absence of the mean-field order parameters has been studied in Refs. [5557,69], showing that any non-magnetic impurity can generally induce a local gap for the SMFBs of DIII TNLS. Our work here aims to present a systematic study on how the LDOS of SMFBs is split around a single non-magnetic or magnetic impurity in the absence/presence of the mean-field order parameters.

5.1. Preliminaries

To consider the local potential, we first need to transform SMFBs to the real space with

where the momentum summation is limited into the surface mode region . Under the symmetry operations, the indexes i,lχ,lc of defined here are transformed in the same way as those of defined in Section 4. In the following, we adopt the following approximation:

resulting in

Further, we define

for convenience.

The behavior of under the symmetry transformation is crucial for the understanding of LDOS. In general, the relation required by the PH symmetry has the form , and the transformation under TR, , and operations reads , and , respectively. As , besides 𝒓, carries three indexes lχ,lc, i that transform independently under the symmetry operation, the transformation matrices presented above should be in the tensor product form as

where , ρi’s are Pauli matrices for lχ = ± index, σi’s are for the pseudo-spin of the surface modes as before, and λi’s are Gell–Mann matrices (Appendix A) for lc = 1,2,3 index with λ0 the 3× 3 identity matrix. In addition, the representation of the translation operator perpendicular to (111) direction is .

With the above definition of operator, we next consider the Hamiltonian that describes the effect of a surface impurity on the SMFBs, given by

where MV(𝒓) is Hermitian, the PH symmetry requires , and the impurity is chosen to be at 𝒓 = 0 without the loss of generality. Such form of impurity Hamiltonian is justified in Appendix F. MV(𝒓) in general is the linear combination of ρjλkσl with coefficients depending on 𝒓. In this case, we can convert the symmetry transformation of lχ and lc indexes of MV(𝒓) to the transformations of ρj’s and λk’s, respectively. Based on Eq. (25), ρj’s and λk’s can be classified according to the IRs of C3v and parities of TR, PH, and χ, as shown in the second lowest and lowest parts of Table 1. Then, the terms in MV(𝒓) with certain symmetry properties can be constructed via the tensor product of the classified ρj’s, λk’s, and σl’s listed in Table 1, which can further determine the number of LDOS peaks. Similar as Section 4, the LDOS discussed here is based on the full spectrum of MV(𝒓), of which only the half with non-negative energy is physical. In the following, we study the LDOS at the impurity position 𝒓 = 0 with the focus on two types of impurities: (i) non-magnetic charge impurity, and (ii) magnetic impurity with magnetization along the (111) direction.

5.2. Non-magnetic charge impurity

For a charge impurity, the potential term MV(𝒓 = 0) = Mc possesses the TR symmetry , the C3v symmetries centered at the impurity with RC3v, and the chiral symmetry (see Appendix F for details). According to its symmetry properties and Table 1, the generic form of Mc reads

where η1,…,8 are real. Below we examine the LDOS on a single charge impurity for SMFBs and compare the case without any order parameter to the cases with A1 (16), A2 (17), and E (18) order parameters. The LDOS around the charge impurity is shown in Figs. 3(a)3(d), which reveal the following features. (1) Since the PH symmetry exists in all the cases, the LDOS is always symmetric with respect to zero energy. (2) If no order parameters exist, there are six peaks (Fig. 3(a)), given by the TR protected double degeneracy of each eigenvalue of Mc according to the Kramer’s degeneracy. (3) In the presence of the A1 order parameter, 8 peaks exist at the impurity (Fig. 3(b)). The reason is the following. Since the translational invariance is absent, the modes with different lχ or lc are coupled by the charge impurity, and the three-fold degeneracy for the pure A1 order parameter case is lifted. Moreover, the appearance of the order parameter breaks the TR symmetry, leaving only the C3v symmetries to protect the degeneracy. For convenience, we choose the eigenstates of rotation as the bases to make the representation C3,d diagonal as

where 1n is the n × n identity matrix. Due to the presence of the A1 order order parameter, the Hamiltonian at the charge impurity becomes Mc + MA1 with MA1 given by transforming Eq. (16) to the d bases (see Appendix F). With the eigen-bases of rotation, Mc + MA1 can be block diagonalized as diag(h1,h2,h3), where h1, h2, and h3 are 4 × 4 Hermitian matrices. With the same bases, the mirror matrix Πd has the form

with

The mirror symmetry gives and , which means the eigenvalues of h1 are the same as those of h3. In fact, the representations of symmetry operations show that the bases of h1 and h3 belong to two-dimensional IRs of C3v while those of h2 belong to one-dimensional IRs of C3v. Therefore, Mc + MA1 has four doubly degenerate and four single eigenvalues, resulting in the 8 LDOS peaks. (4) The 12 LDOS peaks exist at the impurity in the presence of the A2 order parameter (Fig. 3(c)) since the translational invariance and the odd mirror parity of the A2 order parameter are broken by the impurity, and there are no symmetries ensuring any degeneracy. (5) There are 12 LDOS peaks at the impurity for the E order parameter (Fig. 3(d)) because no new symmetries are brought by the impurity. Besides the above five features, the sign change of the charge does not affect the LDOS peaks since the order parameters are all chiral anti-symmetric while the charge impurity is chiral symmetric.

5.3. Magnetic impurity

The potential term MV(𝒓 = 0) = Mm is still Hermitian and PH symmetric for a magnetic impurity with magnetic momentum along (111) direction. Moreover, it is TR-odd , -symmetric , and -odd (Appendix F). According to the symmetry properties and Table 1, the generic form of Mm reads

where η9,…,14 are real. Figures 3(e)3(h) show the LDOS around the magnetic impurity and reveal the following features. (1) The PH symmetry again ensures that the LDOS is always symmetric with respect to zero energy and the E order parameter still has 12 LDOS peaks at the magnetic impurity since no new symmetries appear as shown in Fig. 3(h). (2) If no order parameters exist, there are six peaks (Fig. 3(e)) resulted from the double degeneracy given by the combination of the PH symmetry and odd parity. It is because the combination of the PH symmetry and odd Π parity gives , and since , each eigenvalue of Mm must be doubly degenerate (similar to Kramer’s theorem). (3) The original 4 peaks of the A1 order are splitted into 12 peaks since the magnetic impurity breaks the translational invariance and symmetry (Fig. 3(f)). (4) As shown in Fig. 3(g), the 6 LDOS peaks of the magnetic impurity remain in the presence of the A2 order since the PH symmetry and odd parity are not broken. Besides the above four features, flipping the direction of the magnetic moment, i.e., Mm→–Mm, does not affect the LDOS distribution in presence of the A1 order parameter, since the A1 order parameter has symmetry while Mm has odd parity.

5.4. Summary for impurity effect

To sum up, the number of LDOS peaks at a charge impurity or a magnetic impurity with magnetic moment in (111) direction is 6 or 6 for no order parameters, 8 or 12 for the A1 order parameter, 12 or 6 for the A2 order parameter, and 12 or 12 for the E order parameter, respectively, as summarized in Fig. 4.

Combining the above results with the LDOS peaks without impurity given in Section 4, it is more than enough to identify the order parameters in our system. In the above analysis, we adopt the approximation (22), only consider translationally invariant order parameters that are k-independent in each surface mode region, and assume the surface mode wavefunctions are k-independent in each surface mode region to deal with the impurity. These approximations neglect high-order effects which typically can only broaden the LDOS peaks without affecting the qualitative result.

6. Discussion and conclusion

We studied the energy spectrum (or LDOS) of the SMFBs localized on (111) surface of the half-Heusler SCs with translationally invariant order parameters or magnetic/non-magnetic impurities based on the Luttinger model with singlet–quintet mixing. Our work demonstrates that the zero-bias peak of SMFBs can be split to reveal a rich peak structure when different types of order parameters induced by interaction or magnetic/non-magnetic impurities are introduced. Such peak structure can be viewed as a fingerprint to distinguish different types of order parameters in the standard STM experiments. In addition, we notice that the SMFBs induced by singlet–septet mixing proposed in Ref. [34] possess six patches without any additional pseudospin degeneracy in the surface Brillouin zone (see Fig. 5(a) and the discussion in Ref. [39]). Due to the different number of degeneracy, we expect the peak structures given by the order parameters and magnetic/non-magnetic impurities will be different in the two cases, which thereby may help distinguish the singlet–quintet mixing from the singlet–septet mixing in experiments.

Reference
[1] Kim H Wang K Nakajima Y Hu R Ziemak S Syers P Wang L Hodovanets H Denlinger J D Brydon P M 2018 Science Advances 4 eaao4513
[2] Graf T Parkin S S Felser C 2011 IEEE Trans. Magn. 47 367
[3] Lin H Wray L A Xia Y Xu S Jia S Cava R J Bansil A Hasan M Z 2010 Nat. Materials 9 546
[4] Chadov S Qi X Kübler J Fecher G H Felser C Zhang S C 2010 Nat. Materials 9 541
[5] Xiao D Yao Y Feng W Wen J Zhu W Chen X Q Stocks G M Zhang Z 2010 Phys. Rev. Lett. 105 096404
[6] Al-Sawai W Lin H Markiewicz R S Wray L A Xia Y Xu S Y Hasan M Z Bansil A 2010 Phys. Rev. B 82 125208
[7] Yan B de Visser A 2014 MRS Bull. 39 859
[8] Liu Z K Yang L X Wu S C Shekhar C Jiang J Yang H F Zhang Y Mo S K Hussain Z Yan B Felser C Chen Y L 2016 Nat. Commun. 7 12924 article
[9] Logan J A Patel S J Harrington S D Polley C M Schultz B D Balasubramanian T Palmstrøm C J 2016 Nat. Commun. 7 11993
[10] Cano J Bradlyn B Wang Z Hirschberger M Ong N Bernevig B 2017 Phys. Rev. B 95 161306
[11] Ruan J Jian S K Yao H Zhang H Zhang S C Xing D 2016 Nature Commun. 7 11136
[12] Hirschberger M Kushwaha S Wang Z Gibson Q Liang S Belvin C A Bernevig B A Cava R J Ong N P 2016 Nat Mater 15 1161 ISSN 1476-1122 letter
[13] Shekhar C Nayak A K Singh S Kumar N Wu S C Zhang Y Komarek A C Kampert E Skourski Y Wosnitza J 2016 arXiv:1604.01641 [cond-mat.mtrl-sci]
[14] Suzuki T Chisnell R Devarakonda A Liu Y T Feng W Xiao D Lynn J Checkelsky J 2016 Nat. Phys. 12 1119
[15] Yang H Yu J Parkin S S P Felser C Liu C X Yan B 2017 Phys. Rev. Lett. 119 136401
[16] Liu J Liu H Cao G Zhou Z 2018 arXiv:1808.04748 [cond-mat.mtrl-sci]
[17] Pan Y Nikitin A M Bay T V Huang Y K Paulsen C Yan B H de Visser A 2013 EPL (Europhys. Lett.) 104 27001
[18] Gofryk K Kaczorowski D Plackowski T Leithe-Jasper A Grin Y 2011 Phys. Rev. B 84 035208
[19] Müller R A Lee-Hone N R Lapointe L Ryan D H Pereg-Barnea T Bianchi A D Mozharivskyj Y Flacau R 2014 Phys. Rev. B 90 041109
[20] Nikitin A M Pan Y Mao X Jehee R Araizi G K Huang Y K Paulsen C Wu S C Yan B H de Visser A 2015 J. Phys.: Condens. Matter 27 275701
[21] Nakajima Y Hu R Kirshenbaum K Hughes A Syers P Wang X Wang K Wang R Saha S R Pratt D 2015 Sci. Advances 1 e1500242
[22] Pavlosiuk O Kaczorowski D Fabreges X Gukasov A Wiśniewski P 2016 Sci. Rep. 6 18797
[23] Pavlosiuk O Kaczorowski D Wiśniewski P 2016 Acta Phys. Pol. A 130 573
[24] Yu J Yan B Liu C X 2017 Phys. Rev. B 95 235158
[25] Pavlosiuk O Fabreges X Gukasov A Meven M Kaczorowski D Wiśniewski P 2018 Physica B 536 56
[26] Goll G Marz M Hamann A Tomanic T Grube K Yoshino T Takabatake T 2008 Physica B 403 1065
[27] Butch N P Syers P Kirshenbaum K Hope A P Paglione J 2011 Phys. Rev. B 84 220504
[28] Bay T V Naka T Huang Y K de Visser A 2012 Phys. Rev. B 86 064515
[29] Tafti F F Fujii T Juneau-Fecteau A René de Cotret S Doiron-Leyraud N Asamitsu A Taillefer L 2013 Phys. Rev. B 87 184504
[30] Xu G Wang W Zhang X Du Y Liu E Wang S Wu G Liu Z Zhang X X 2014 Sci. Rep. 4 5709
[31] Pavlosiuk O Kaczorowski D Wiśniewski P 2015 Sci. Rep. 5 9158
[32] Meinert M 2016 Phys. Rev. Lett. 116 137001
[33] Xiao H Hu T Liu W Zhu Y L Li P G Mu G Su J Li K Mao Z Q 2018 Phys. Rev. B 97 224511
[34] Brydon P M R Wang L Weinert M Agterberg D F 2016 Phys. Rev. Lett. 116 177001
[35] Kawakami T Okamura T Kobayashi S Sato M 2018 Phys. Rev. X 8 041026
[36] Wu C 2006 Mod. Phys. Lett. B 20 1707
[37] Kuzmenko I Kuzmenko T Avishai Y Sato M 2018 Phys. Rev. B 98 165139
[38] Yang W Xiang T Wu C 2017 Phys. Rev. B 96 144514
[39] Timm C Schnyder A P Agterberg D F Brydon P M R 2017 Phys. Rev. B 96 094526
[40] Yu J Liu C X 2018 Phys. Rev. B 98 104514
[41] Wang Q Z Yu J Liu C X 2018 Phys. Rev. B 97 224507
[42] Yu J Liu C X 2018 arXiv:1809.04736 [cond-mat.supr-con]
[43] Roy B Ghorashi S A A Foster M S Nevidomskyy A H 2018 Phys. Rev. B 99 054505
[44] Boettcher I Herbut I F 2018 Phys. Rev. Lett. 120 057002
[45] Yang W Li Y Wu C 2016 Phys. Rev. Lett. 117 075301
[46] Venderbos J W F Savary L Ruhman J Lee P A Fu L 2018 Phys. Rev. X 8 011029
[47] Savary L Ruhman J Venderbos J W F Fu L Lee P A 2017 Phys. Rev. B 96 214514
[48] Ghorashi S A A Davis S Foster M S 2017 Phys. Rev. B 95 144503
[49] Brydon P Agterberg D Menke H Timm C 2018 Phys. Rev. B 98 224509
[50] Yada K Sato M Tanaka Y Yokoyama T 2011 Phys. Rev. B 83 064505
[51] Li Y Wang D Wu C 2013 New J. Phys. 15 085002
[52] Potter A C Lee P A 2014 Phys. Rev. Lett. 112 117002
[53] Timm C Rex S Brydon P M R 2015 Phys. Rev. B 91 180503
[54] Hofmann J S Assaad F F Schnyder A P 2016 Phys. Rev. B 93 201116
[55] Ikegaya S Asano Y Tanaka Y 2015 Phys. Rev. B 91 174511
[56] Ikegaya S Asano Y 2017 Phys. Rev. B 95 214503
[57] Ikegaya S Kobayashi S Asano Y 2018 Phys. Rev. B 97 174501
[58] Luttinger J M 1956 Phys. Rev. 102 1030
[59] Winkler R Papadakis S De Poortere E Shayegan M 2003 Spin-Orbit Coupling Two-Dimensional Electron. Hole Syst. 41 Berlin Springer
[60] Blount E I 1985 Phys. Rev. B 32 2935
[61] Ueda K Rice T M 1985 Phys. Rev. B 31 7114
[62] Volovik G Gorkov L 1985 Zh. Eksperimentalnoi I Teor. Fiz. 88 1412
[63] Sigrist M Ueda K 1991 Rev. Mod. Phys. 63 239
[64] Annett J F 1990 Adv. Phys. 39 83
[65] Annett J Goldenfeld N Renn S R 1991 Phys. Rev. B 43 2778
[66] Annett J F Goldenfeld N Leggett A J 1996 J. Low Temp. Phys. 105 473
[67] Tinkham M 1996 Introduction to superconductivity New York McGraw-Hill
[68] Bi Z Yuan N F Q Fu L 2019 Phys. Rev. B 100 035448
[69] Sato M Tanaka Y Yada K Yokoyama T 2011 Phys. Rev. B 83 224511
[70] Murakami S Nagosa N Zhang S C 2004 Phys. Rev. B 69 235206
[71] Aroyo M I Kirov A Capillas C Perez-Mato J Wondratschek H 2006 Acta Crystallogr. Sect. A 62 115
[72] Gell-Mann M 1962 Phys. Rev. 125 1067